An automated calibration method for non-see-through head mounted displays

Accurate calibration of a head mounted display (HMD) is essential both for research on the visual system and for realistic interaction with virtual objects. Yet, existing calibration methods are time consuming and depend on human judgements, making them error prone, and are often limited to optical see-through HMDs. Building on our existing approach to HMD calibration Gilson et al. (2008), we show here how it is possible to calibrate a non-see-through HMD. A camera is placed inside a HMD displaying an image of a regular grid, which is captured by the camera. The HMD is then removed and the camera, which remains fixed in position, is used to capture images of a tracked calibration object in multiple positions. The centroids of the markers on the calibration object are recovered and their locations re-expressed in relation to the HMD grid. This allows established camera calibration techniques to be used to recover estimates of the HMD display's intrinsic parameters (width, height, focal length) and extrinsic parameters (optic centre and orientation of the principal ray). We calibrated a HMD in this manner and report the magnitude of the errors between real image features and reprojected features. Our calibration method produces low reprojection errors without the need for error-prone human judgements

View-based approaches to spatial representation in human vision

In an immersive virtual environment, observers fail to notice the expansion of a room around them and consequently make gross errors when comparing the size of objects. This result is difficult to explain if the visual system continuously generates a 3-D model of the scene based on known baseline information from interocular separation or proprioception as the observer walks. An alternative is that observers use view-based methods to guide their actions and to represent the spatial layout of the scene. In this case, they may have an expectation of the images they will receive but be insensitive to the rate at which images arrive as they walk. We describe the way in which the eye movement strategy of animals simplifies motion processing if their goal is to move towards a desired image and discuss dorsal and ventral stream processing of moving images in that context. Although many questions about view-based approaches to scene representation remain unanswered, the solutions are likely to be highly relevant to understanding biological 3-D vision

No single, stable 3D representation can explain pointing biases in a spatial updating task

People are able to keep track of objects as they navigate through space, even when objects are out of sight. This requires some kind of representation of the scene and of the observer's location but the form this might take is debated. We tested the accuracy and reliability of observers' estimates of the visual direction of previously-viewed targets. Participants viewed 4 objects from one location, with binocular vision and small head movements then, without any further sight of the targets, they walked to another location and pointed towards them. All conditions were tested in an immersive virtual environment and some were also carried out in a real scene. Participants made large, consistent pointing errors that are poorly explained by any stable 3D representation. Any explanation based on a 3D representation would have to posit a different layout of the remembered scene depending on the orientation of the obscuring wall at the moment the participant points. Our data show that the mechanisms for updating visual direction of unseen targets are not based on a stable 3D model of the scene, even a distorted one

Comparison of view-based and reconstruction-based models of human navigational strategy

There is good evidence that simple animals such as bees use view-based strategies to return to a familiar location but humans could use a 3D reconstruction to achieve the same goal. Assuming some noise in the storage and retrieval process, these two types of strategy give rise to different patterns of predicted errors in homing. We describe an experiment that can help distinguish between these models. Participants wore a head mounted display to carry out a homing task in immersive virtual reality. They viewed three long thin vertical poles and had to remember where they were in relation to the poles before being transported (virtually) to a new location in the scene from where they had to walk back to the original location. The experiment was conducted in both a rich-cue scene (a furnished room) and a sparse scene (no background and no floor or ceiling). As one would expect, in a rich-cue environment the overall error was smaller and in this case the ability to separate the models was reduced. However, for the sparse-cue environment the view-based model outperforms the reconstruction-based model. Specifically, the likelihood of the experimental data is similar to the likelihood of samples drawn from the view-based model (but assessed under both models) while this is not true for samples drawn from the reconstruction-based model

Markerless Tracking using Planar Structures in the Scene

Colloque avec actes et comité de lecture. internationale.International audienceWe describe a markerless camera tracking system for augmented reality that operates in environments which contain one or more planes. This is a common special case, which we show significantly simplifies tracking. The result is a practical, reliable, vision-based tracker. Furthermore, the tracked plane imposes a natural reference frame, so that the alignment of the real and virtual coordinate systems is rather simpler than would be the case with a general structure-and-motion system. Multiple planes can be tracked, and additional data such as 2D point tracks are easily incorporated

Spatial calibration of an optical see-through head-mounted display

We present here a method for calibrating an optical see-through Head Mounted Display (HMD) using techniques usually applied to camera calibration (photogrammetry). Using a camera placed inside the HMD to take pictures simultaneously of a tracked object and features in the HMD display, we could exploit established camera calibration techniques to recover both the intrinsic and extrinsic properties of the~HMD (width, height, focal length, optic centre and principal ray of the display). Our method gives low re-projection errors and, unlike existing methods, involves no time-consuming and error-prone human measurements, nor any prior estimates about the HMD geometry

Visual Experiences during Paralysis

Rationale: Paralyzed human volunteers (n = 6) participated in several studies the primary one of which required full neuromuscular paralysis while awake. After the primary experiment, while still paralyzed and awake, subjects undertook studies of humor and of attempted eye-movement. The attempted eye-movements tested a central, intentional component to one’s internal visual model and are the subject of this report. Methods: Subjects reclined in a supportive chair and were ventilated after paralysis (cisatracurium, 20 mg intravenously). In illumination, subjects were requested to focus alternately on the faces of investigators standing on the left and the right within peripheral vision. In darkness, subjects were instructed to look away from a point source of light. Subjects were to report their experiences after reversal of paralysis. Results: During attempted eye-movement in illumination, one subject had an illusion of environmental movement but four subjects perceived faces as clearly as if they were in central vision. In darkness, four subjects reported movement of the target light in the direction of attempted eye-movements and three could control the movement of the light at will. Conclusion: The hypothesis that internal visual models receive intended ocular-movement-information directly from oculomotor centers is strengthened by this evidence

Visual Experiences during Paralysis

Rationale: Paralyzed human volunteers (n = 6) participated in several studies the primary one of which required full neuromuscular paralysis while awake. After the primary experiment, while still paralyzed and awake, subjects undertook studies of humor and of attempted eye-movement. The attempted eye-movements tested a central, intentional component to one’s internal visual model and are the subject of this report. Methods: Subjects reclined in a supportive chair and were ventilated after paralysis (cisatracurium, 20 mg intravenously). In illumination, subjects were requested to focus alternately on the faces of investigators standing on the left and the right within peripheral vision. In darkness, subjects were instructed to look away from a point source of light. Subjects were to report their experiences after reversal of paralysis. Results: During attempted eye-movement in illumination, one subject had an illusion of environmental movement but four subjects perceived faces as clearly as if they were in central vision. In darkness, four subjects reported movement of the target light in the direction of attempted eye-movements and three could control the movement of the light at will. Conclusion: The hypothesis that internal visual models receive intended ocular-movement-information directly from oculomotor centers is strengthened by this evidence

Modelling human visual navigation using multi-view scene reconstruction

It is often assumed that humans generate a 3D reconstruction of the environment, either in egocentric or world-based coordinates, but the steps involved are unknown. Here, we propose two reconstruction-based models, evaluated using data from two tasks in immersive virtual reality. We model the observer’s prediction of landmark location based on standard photogrammetric methods and then combine location predictions to compute likelihood maps of navigation behaviour. In one model, each scene point is treated independently in the reconstruction; in the other, the pertinent variable is the spatial relationship between pairs of points. Participants viewed a simple environment from one location, were transported (virtually) to another part of the scene and were asked to navigate back. Error distributions varied substantially with changes in scene layout; we compared these directly with the likelihood maps to quantify the success of the models. We also measured error distributions when participants manipulated the location of a landmark to match the preceding interval, providing a direct test of the landmark-location stage of the navigation models. Models such as this, which start with scenes and end with a probabilistic prediction of behaviour, are likely to be increasingly useful for understanding 3D vision